|
Publication Date: 8/1/75
Pages: 6 Date Entered: 1/5/93 Title: Qualification, Calibration, and Error Estimation Methods for Nondestructive Assay August 1975 U.S. NUCLEAR REGULATORY COMMISSION REGULATORY GUIDE OFFICE OF STANDARDS DEVELOPMENT REGULATORY GUIDE 5.53 QUALIFICATION, CALIBRATION, AND ERROR ESTIMATION METHODS FOR NONDESTRUCTIVE ASSAY A. INTRODUCTION Section 70.58, "Fundamental Nuclear Material Controls," of 10 CFR Part 70, "Special Nuclear Material," requires certain licensees to establish a measurement quality assurance program for material control and accounting. Specifically, paragraph 70.58(f) requires that a program be established, maintained, and followed for the maintenance of acceptable measurement quality in terms of measurement bias and random and systematic errors and for the evaluation and control of the quality of the measurement system. Nondestructive assay (NDA) constitutes a unique measurement technology. When applied under appropriate rigorous controls, it can enhance the ability of the material control and accounting system to detect diversion of special nuclear material to unauthorized uses. This guide describes methods and procedures acceptable to the NRC staff for meeting the provisions of paragraph 70.58(f) of 10 CFR Part 70 as it relates to the use of nondestructive assay. B. DISCUSSION Nondestructive assay has been applied to virtually every variety of chemical or physical form of special nuclear material encountered in contemporary reactor fuel processing. Special considerations are required to achieve high-accuracy assay results and to properly estimate the errors associated with nondestructive assay applications. Recognizing these considerations, the American National Standards Institute has developed Standard N15.20-1975, entitled "Guide to Calibrating Nondestructive Assay Systems."(1) This guide endorses the standard as augmented in the regulatory position. C. REGULATORY POSITION The methods, procedures, and guidance relating to the application of nondestructive assay, as detailed in the American National Standards Institute Standard N15.20-1975, "Guide to Calibrating Nondestructive Assay Systems," are acceptable to the NRC staff for material protection programs as supplemented by the following. 1. Nondestructive Assay Method Selection Prior to selection of an assay method, a study should be made to determine the required performance for that application. The specific nondestructive assay method should be selected to provide precise and accurate results which are compatible with plant material balance requirements. Methods to enhance attainable performance should be considered (e.g., container selection and packaging procedures for bulk materials). (See also Ref. 1.)2. Instrument Specifications An evaluation of each new nondestructive assay application, including the proposed placement of the instrument, should be conducted prior to procurement. A study of each existing nondestructive assay application should be conducted to substantiate the basis for its further use. The impact of each of the measurement-to-measurement and item-to-item sources of error encountered in practice or anticipated should be established as a part of each of these efforts. ---------- (1) Copies may be obtained from the American National Standards Institute, 1430 Broadway, New York, New York 10018. ---------- A decision should be made to reduce each potentially significant source of error through (1) appropriate instrument design considerations, (2) operational controls, or (3) supplementary measurements made to establish bias corrections (see also Ref. 2). Instrument procurement specifications and operational instructions should be developed and followed to reflect each error reduction decision. To minimize operator-related errors and to promote uniform measurement practices, nondestructive assay instruments used for fixed-station operations should be automated to control (1) data acquisition and analysis, (2) diagnostic testing of instrument performance stability and calibration validity, and (3) calculation of associated error estimates. Instruments should be tested to ensure meeting procurement specifications prior to calibration. 3. Operators Operators should be selected, trained, and tested to be qualified before routine use of the instrument. Only qualified operators should be permitted to make special nuclear material assays. 4. Stability Testing A preventive maintenance program should be devised and implemented to ensure the long-term stability and reliability of each instrument. As part of an on-going program, a minimum of two working standards(2) should be fabricated to periodically test the performance stability of the instrument. Each working standard should contain a different amount of the species of the special nuclear material to be assayed. Working standards should be nominally representative of the items to be assayed, as described in ANSI N15.20-1975. ---------- (2) Working standards are used to check the performance of an NDA instrument. They should be nominally representative of the items to be assayed. They should be fabricated and handled to ensure their internal integrity so that deviations in the measured response of the assay system can be attributed to the instrument. As stated in ANSI N15.20-1975, working standards built to meet these requirements are not acceptable as calibration standards. Calibration standards are defined in ANSI N15.20-1975 as "physically and chemically similar to the items to be assayed, for which the mass of the nuclide(s) of interest and all properties to which the measurement technique is sensitive are known." Calibration standards can be used as working standards, but working standards cannot be used as calibration standards. When calibration standards meet the requirements for working standards, licensees may elect to maintain only calibration standards. In many cases, however, calibration standards may deteriorate through extensive use, or may be prohibitively expensive for stability monitoring purposes. ---------- A study should be made to determine the frequency with which the working standards are to be measured. In cases of extreme instability, a working standard should be measured before and after each unknown item assay, and the calibration should be normalized to reflect the average of the before-assay and after-assay tests. In any case, a working standard should be measured a minimum of twice per shift, once at the beginning of the shift and again at some random time during the shift. Each response to a working standard should be compared to the mean value of all previous measurements of that working standard that were accumulated during the preceding material balance period; the difference should be plotted on a working chart. Control chart limits should be established at 0.05 and 0.001 levels of significance. Whenever control data exceed the 0.05 control limits, the test should be repeated. Whenever the control data exceed the 0.001 control limits, normal assay operations should cease. Normal operations should not resume until the out-of-control performance has been remedied and the instrument recalibrated. The control chart of the working standard responses should be examined at frequent intervals to detect indications of drift, which should be compensated. The frequency for such examinations should be determined by the operating characteristics of each instrument. A minimum frequency for examining the control chart for indications of drift should be once per week. 5. Calibration Nondestructive assay should be employed as a relative indicator of special nuclear material content. Thus, calibration should be accomplished by measuring the response to calibration standards whose contents are characterized through chemical assay procedures, as described in ANSI N15.20-1975. The chemical assay procedures should be calibrated relative to national standards or nationally accepted measurement systems. The calibration standards should represent the unknown items in both physical and chemical characteristics. Calibration data should be obtained by averaging the responses from repeated measurements of the calibration standards. Calibration data should be corrected to remove measured nonrandom variations. Recalibration should be performed following any repair or parts replacement or whenever the characteristics of the items being assayed shift so that previous calibration standards no longer adequately represent the unknown items. The calibration should be checked following a power outage or any unusual mechanical or electrical shock to the system. Criteria for segregating and packaging different forms of special nuclear material should be developed and implemented. Each material category should be established to enhance assay performance, consistent with safety requirements and subsequent processing needs. (Guidance for material categorization is provided in Regulatory Guides 5.2, "Classification of Unirradiated Plutonium and Uranium Scrap;" 5.11, "Nondestructive Assay of Special Nuclear Material Contained in Scrap and Waste;" and 5.34, "Nondestructive Assay for Plutonium in Scrap Material by Spontaneous Fission Detection.") For all categories of materials to be assayed, with the exception of small content miscellaneous categories (e.g., furnace liner bricks, contaminated tools or machine parts), the calibration relationship should be a least-squares fit to an appropriate function. The graphical calibration method is acceptable for miscellaneous categories of material which contain a total of no more than 0.1 effective kilogram(3) of special nuclear material in each category during a material balance period. The combined contribution from all assays calibrated through the graphical method should be less than 10% of the total plant limit of error of material unaccounted for (LEMUF). 6. Calibration Standards A minimum of four calibration standards should be obtained to serve as the basis for the initial calibration of each instrument for each separate category of material. The calibration standards should be completely characterized, including the mass and isotopic composition of the species of special nuclear material to be assayed, plus all physical or chemical variables to which the response of the instrument is sensitive. The mass of special nuclear material contained in the standards should span the range of loadings to be encountered in routine assays. The isotopic composition of the material used in all calibration standards should be identical with the isotopic composition of the material being assayed. When the isotopic composition changes so that the response per gram of special nuclear material differs by 10% or more from the value of the calibration standards, the material should be identified as a new material category. The nondestructive assay system should be recalibrated for that category, using new calibration standards made up using the new isotopic composition. When the change in response per gram is less than 10%, a bias correction should be determined and applied to the assay data. ---------- (3) The term "effective kilogram" is defined in 10 CFR Section 70.4(t). ---------- The uncertainty in the bias correction should be determined and used in estimating the total assay uncertainty. (Appropriate error propagation procedures are described in Regulatory Guide 5.18, "Limit of Error Concepts and Principles of Calculation in Nuclear Materials Control.") When the response is sensitive to ingrowth or decay of a daughter product, the procedures described in the preceding paragraphs are appropriate and should be applied. Once fabricated, the calibration standards should be handled with extreme care to ensure that the distribution of contents remains fixed. Calibration standards should be used only when developing the initial calibration or when recalibrating the instrument following a repair or power outage. (Working standards should be used to test the continued stability of the instrument, see footnote 2.) The degree of effort which should be expended in fabricating the calibration standards depends on the method used to estimate the assay uncertainty, as described in the next section. 7. Assay Uncertainty Estimation Methods Random errors associated with nondestructive assay should be estimated through a replicate assay program during each material balance period.(4) The second assay of each replicate pair should be performed a minimum of 4 hours after the first assay. Three methods are acceptable to estimate the uncertainties associated with calibrations and bias corrections for nondestructive assay. The first two procedures, graphical estimation and analytical estimation through the calibration relationship, are detailed in the standard. The third procedure, comparative evaluation, is not described in ANSI N15.20-1975. ---------- (4) A regulatory guide related to random error evaluation is in preparation (to be entitled "Considerations for Determining the Random Error of Special Nuclear Material Accounting Measurements"). ---------- 7.1 Graphical Estimation Use of the graphical error estimation technique should result in a conservative error estimate that is acceptable for miscellaneous unusual assay categories, as described in Regulatory Position 5 of this guide. 7.2 Analytical Estimation Through the Calibration Relationship When the calibration standards can be shown to adequately represent the unknown items, the combined systematic error associated with the nondestructive assay of an inventory of items can be estimated through the calibration relationship, as demonstrated in ANSI N15.20-1975. The calibration standards should be fabricated from different batches of material. The uncertainty associated with the content of special nuclear material elements and response-related isotopes contained in each calibration standard should be based on an extensive characterization as described in N15.20. The uncertainty associated with the contained mass of the response-related isotopes should be included in the calibration, as described in N15.20. Further, the element uncertainty should be factored into the estimated total assay uncertainty. Using this procedure, it is necessary to periodically ensure that the calibration standards adequately represent the unknown items. This can be accomplished by isolating and characterizing the extraneous interference factors that affect the response of the instrument. Typically, this separation and characterization is most easily accomplished when the items are either finished fuel items or uniform containers of feed or intermediate product material. To ensure that the calibration standards continue to adequately represent the unknown items, key parameters(5) that affect the observed response (i.e., item-to-item variations) should be monitored through separate tests. Measurements of the key parameters should be compiled and analyzed at the close of each material balance period. When the mean value of a parameter shifts from its previously established value, the impact of the shift on the response of the assay instrument should be measured through an appropriate experiment or calculation (Ref. 3). An appropriate bias correction should be determined and applied to all items that were assayed after the best estimate of when the parameter changed. The uncertainty in that bias estimate should be combined with the uncertainty in the assay values as predicted through the calibration function to estimate the total assay uncertainty. ---------- (5) See Section 5.4 of ANSI N15.20-1975. See Regulatory Position 6 of this guide for provisions to include the effects of changing isotopic compositions. ---------- The uncertainty due to a bias correction may significantly increase the limit of error of the assay. In severe cases, the increase may increase the LEMUF above the level acceptable for the total plant. In such cases, new calibration standards should be obtained and the assay system should be recalibrated. As a further check on the continued validity of the calibration standards, a program to periodically introduce new calibration standards should be implemented. A minimum of one new calibration standard. for each category of materials should be introduced during each 6-month period. 7.3 Comparative Evaluation The procedure described in this section is not included in ANSI N15.20-1975. When two measurement methods are used for each of a series of items and one of the methods is considerably more accurate than the other, corresponding measurements can be usefully compared. The comparison can be used to establish an estimate of bias between the measurement methods. The comparison can also be used to estimate the total uncertainty associated with the less accurate measurement method. To determine the uncertainty associated with the nondestructive assay of an inventory of items using this method, unknown items should be randomly selected for comparative measurements. The special nuclear material content of the items selected should span the range of contents normally encountered. Random error should be estimated through replicate analyses. To estimate the remaining contributions to the total assay uncertainty, each item should be repeatedly assayed to reduce the random assay error to less than 10% of the estimated or previously established total uncertainty. Then, to determine the special nuclear material content of each item selected for comparative evaluation, one of the following procedures should be employed: a. Each item should be completely dissolved, independently, and the resulting solution should be analyzed by high-accuracy elemental and isotopic assay procedures, which in turn are calibrated relative to national standards or nationally accepted measurement systems. Items composed of an aggregate of similar units, e.g., fuel rods containing discrete pellets, should be opened and the contained units should be weighed, pulverized, blended, and sampled for assay through appropriate high-accuracy elemental and isotopic assay procedures. The emptied container should be examined for indications of residual accumulations and cleaned, leached, or assayed nondestructively to determine the residual special nuclear material content. b. Plutonium-bearing items only: Each item should be assayed through calorimetric procedures (Ref. 4). Large items should be subdivided into smaller containers. Each small container should be assayed calorimetrically. Samples should be taken from at least three of the smaller containers. The samples should be micro-calorimetered and then assayed through highly accurate elemental and isotopic procedures which in turn are calibrated relative to national standards or nationally accepted measurement systems (Ref. 5). The istotopic measurement data should be examined for evidence of nonhomogeneous isotopic content. Isotopically nonhomogeneous materials should be blended and reanalyzed. On the basis of the average grams of plutonium per watt of the micro-calorimetered samples, the total amount of plutonium in each of the subcontainers should be determined. The total plutonium content of the items selected for comparison is then estimated as the combined contents of the subcontainers. For the first full material balance period during the initial implementation of this guide, two items from each category of assay items should be randomly selected each week for an accuracy verification measurement. Following this initial implementation period, licensees may reduce the verification measurement frequency to two items per month per category. When fewer than one hundred new items of a given category are created per week, at least two item comparative verification measurements should be made per material balance period per category, through the procedures described above. In such cases, to provide an adequate data base to update the uncertainty estimates for nondestructive assay, licensees may pool the verification data provided the measurements are in statistical control; i.e., when repeated samples from the portion of the measurement system under test behave as random samples from a stable probability distribution. Under such conditions, data sets may be combined provided the parameters based on the current set of data and the previous set of data are not significantly different on the basis of acceptable statistical tests. As an alternative to this selection criteria, licensees may elect the latter frequency for a specific category when it can be demonstrated that the contribution to the LEMUF from that category is less than 100 grams in any material balance period. At the close of the reporting period, the assay value for each item selected for verification should be plotted against the verified quantity. The verification data plot should be examined for indications of nonlinearity or obvious outlier data. Anomalous indications should be investigated and remedied. A linear regression analysis to a non-zero-intercept straight line should be performed on the nondestructive assay versus verification assay data. The intercept should be tested against zero with a "t" test at a 5% level of significance for an indication of a constant bias. In addition, the slope should be tested against unity for an indication of proportional bias. When bias is indicated (i.e., the bias exceeds 10% of its estimated uncertainty), assays performed during the preceding operating period should be compensated. The uncertainty associated with bias corrections should be estimated with the standard error of estimate associated with the verification line. The total assay uncertainty should be estimated by combining the random error estimate for the collection of items assayed with the estimated bias uncertainty. When all items contain essentially the same special nuclear material content, the difference in the mean content values should be tested against zero as an indication of bias. The total assay uncertainty associated with an inventory of items should be estimated as the standard deviation of the mean difference, combined with the random assay error. For individual items, the assay uncertainty should be estimated as the standard deviation of the difference distribution, combined with the estimated single measurement random error. Again, bias corrections should be applied when the indicated bias exceeds 10% of the standard deviation of the mean difference. Whenever a bias exceeding 50% of its estimated uncertainty is indicated, its cause should be investigated. This investigation should include a review of the assumptions factored into the nondestructive assay system's calibration. In particular, instrument stability and the stability of parameters that may influence the response of the assay system should be investigated. The investigation should also address the comparative measurement method, including sampling, sample handling, analytical procedures, interference compensation, and calibration validity. Results from the investigation, if they show the nondestructive assay system to have been incorrectly calibrated, should be employed to recalibrate the instrument for the forthcoming material balance period. Conversely, when the source of bias can be attributed to errors in the comparative measurements, bias corrections should not be made to the nondestructively assayed items. Results from such investigations should be documented, and the documents maintained in accordance with Regulatory Position 8 of this guide. 8. Records Retention All records generated in accordance with this guide, including control charts, should be retained for a period of 5 years, as specified in 10 CFR Section 70.51 (e)(4)(iii). D. IMPLEMENTATION This section provides information to applicants and licensees regarding the NRC staff's plans for using this regulatory guide. Except in those cases in which the applicant or licensee proposes an alternative method, the NRC staff will use the methods described herein in evaluating an applicant's or licensee's capability for and performance in complying with specified portions of the Commission's regulations for submittals docketed after October 1, 1975, in connection with a current license or an application for a license, or amendment of a current license. An applicant or licensee may use this regulatory guide in developing submittals docketed on or before October 1, 1975. The pertinent portions of these submittals will be evaluated on the basis of this guide. REFERENCES 1. Regulatory Guide 5.11, "Nondestructive Assay of Special Nuclear Material Contained in Scrap and Waste," U.S. Atomic Energy Commission (1973). 2. T. E. Shea, "Reduction, Control, and Estimation of Nondestructive Assay Errors," Nucl. Mat. Mgmt., Vol. III, No. 3 (1974). 3. R. A. Forster, D. B. Smith, and H. O. Menlove, "Error Analysis of a Cf-252 Fuel-Rod-Assay System," LASL Reports LA-5317 (1974). 4. Regulatory Guide 5.35, "Calorimetric Assay of Plutonium," U.S. Atomic Energy Commission (1974). 5. F. S. Stephens, R. G. Gutmacher, K. Ernst, S. P. Turel, and T. E. Shea, "Methods for the Accountability of Plutonium Dioxide," WASH-1335 (1975). 44 |